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ABSTRACT 



In the military, there has been much interest in 
combining instructional methodologies with computer 
technology to facilitate training* The potential for life 
cycle cost reduction while improving the effectiveness of 
training is very attractive. Thus, Computer-Based Training 
(CBT) has been embraced by the U . S. Navy. Initially, CBT 
was used primarily to manage training programs and has more 
recently also been used to provide instruction directly. 
However, CBT has not adequately kept pace with the 
formidable record-keeping and data requirements of a typical 
Navy training program. Proposals are introduced here to 
improve data processing and management for CBT. Also, 
concepts are described to promote better decision-making so 
that more efficient use can be made of CBT resources, both 
now and in the future. Methods to facilitate actual 
training with CBT are discussed. These initiatives are 
applied specifically to the Navy Maintenance Training 
Improvement Program (MTIP). 
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I. INTRODUCTION 



A . COMPUTER-BASED TRAINING 

In recent decades, there has been significant interest 
in using individualized instruction and computer technology 
to facilitate training. The potential cost reduction and 
learning efficiency advantages seemed readily apparent. 
Hence, much research and development has been directed to 
that area (see Appendix C). A widely used methodology for 
conducting training has evolved. The term to be used here 
for that activity is Computer-Based Training (CBT). CBT is 
comprised of two major subsystems, Computer-Managed 
Instruction (CMI) and Computer-Aided Instruction (CAI). 
[O’Neil 81:p. 1] CAI is also frequently referred to as 

Computer-Aided Learning (CAL). CBT, or Computer-Based 
Instruction (CBI) as it is frequently known, is primarily 
comprised of and loosely defined as the sum of these two 
components . 

CBT, CMI and CAI are selected from a proliferation of 
terminology which has developed over the years. Appendix A 
provides definitions of most associated terms found in 
current literature. Appendix B lists many CBT and U. S. 
Navy-related acronyms. 

CBT development typically begins with the CMI component. 
CMI is defined as a subsystem of CBT which uses the computer 
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to manage overall instructional activity. This includes 
planning, administration, monitoring student progress, 
allocating instructional resources and providing reports, 

CMI is the function which regulates the level of control and 
efficiency in the training system. Figure 1,1 illustrates 
some basic components of a typical training center as they 
are related to a basic CMI subsystem. Notice that CMI 
accommodates both non-computer ized and computerized 
instruction , 

CAI, the other major subsystem of CBT, is defined as a 
method of learning where a computer is the primary storage 
and delivery device for instruction, like a tutor. The 
student interacts directly with a computer in an 
individualized training environment. The student works in 
real time and may receive direct feedback from the 
computer, [NPRDC TN 82-17, 82:p, 2; Burke 82:p, 187] 

CMI and CAI are not entirely distinct and disjoint 
components as illustrated in Figure 1,2, The primary areas 
of commonalty are control functions and the hardware used, 
CMI provides overall external control for CAI, However, 
internally CAI provides control for its own processes in the 
form of an operating system and externally must interface 
with the CMI system. 

Material presented by CMI is generally not instructional 
but rather related to administrative tasks and evaluation 
processes, such as testing or collecting statistical 
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Figure 1*1 Major Components of a Training Center 

[ Baker 81 : p . 28 ] 

information for a database or a report. CMI is not 
necessarily conducted in real time and students and 
instructor generally share a few workstations. CMI is 
inherently of more benefit to the supervisor of training or 
the instructors. This is because it simplifies many of the 
routine administrative facets of their jobs, freeing them to 
spend more instructional time with trainees. 



3 




CBT = Computer-Based Training 

CM I = Computer-Managed Instruction 

CAI = Computer-Aided Instruction 

Figure 1.2 Components of Computer-Based Training 

[Dean 83:p. 132] 

CAI is instructional in nature and can provide intensive 
training. More workstations are required per student 
because of its interactive and individualized character. 
Trainees inherently receive the most benefit from the 
tutorial nature of CAI, 
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B. NEED FOR RESEARCH 



There are various ways to create and present instruction 
and training. Each way highlights different aspects of the 
teaching-learning process. One way depends on a "master" 
teacher. The teacher, working alone, creates material or 
chooses content and presents training directly to students. 
Some historical examples of great teachers are Buddha, 
Confucius, Jesus, Mohammed and Plato. However, a great 
teacher is very rare indeed and generation of original 
training ends with his or her death. 

A second way focuses on the student. The teacher 
provides an environment for learning but the student must 
take the initiative and is primarily responsible for 
learning. This has the distinct disadvantage of allowing a 
long or uncertain period of time to attain a specified level 
of skill or knowledge. Finally, the student "may not be 
able to get there from here" without intervention from a 
teacher or other external stimuli. 

Another way relies on "expert" logic. A problem is 
defined, a solution is proposed and implemented, data 
gathered from the resulting training process and the results 
analyzed. The analysis and feedback are used to make 
revisions for improving the training process. Once a good 
"formula" for effective training is found, it can be 
replicated or adapted to other uses. As a practical matter. 
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this systems approach is applied to virtually all training 
programs . 

Instructional Systems Development (ISD) is a process 
which provides a general systems approach with multiple 
components, operating within a certain set of constraints, 
to produce an instructional system. [Logan 82:pp. 1-3] CBT 
is frequently generated with ISD. 

CBT has often been applied directly to existing training 
methods. Just as there is no "cookbook” solution for ISD, 
there is no "best" way to do CBT. As a consequence, for 
most existing CBT, instruction could be done satisfactorily 
without a computer. This is not to say that CBT is 
ineffective. In certain situations, CBT can be and is very 
appropriate. [NPRDC TR 84-54, 84:pp. 4-19] 

This research integrates computer-based processing and 
training objectives with Database Management System (DBMS), 
Decision Support System (DSS) and Expert System concepts. 

The focus is on CBT development and implementation, for the 
Navy’s Maintenance Training Improvement Program (MTIP). CBT 
development has not adequately recognized the formidable 
record-keeping requirements of a typical military training 
program. Thousands of learning objectives must be 
developed, cross-referenced, tested and taught. 

Additionally, CBT streamlines record-keeping and facilitates 
the CBT process to provide management information for better 
evaluation and monitoring. CBT users need the ability to 
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customize and adapt CBT to provide the best training 
possible. [NPRDC TR 84-54, 84:pp. 10-11] 

C. BENEFITS OF STUDY 

Training innovations have been described as having an 
unusual three-stage life cycle. In the first stage, 
advocates of the innovation proclaim its usefulness and 
success. CBT may be considered such an innovation. In 
stage two, indiscriminate use becomes widespread, often 
without sufficient development, planning or training for 
intended users. In the final stage, the innovation’s 
shortcomings become apparent from experience. Criticism and 
skepticism grow, but come too late in the process to help 
and hasten abandonment of the technique in favor of another 
new innovation. The cycle begins anew. [NPRDC TR 84-54, 

84 : p . 14 ] 

A benefit of this study is a contribution to breaking or 
modifying the cycle for CBT applied to training programs 
such as MTIP. CBT must be thought of as a combination of 
learning theory and computer technology. The best CBT; 
however, results from the development and recognition of CBT 
itself as a new technology. Ultimately, this improves the 
methods and processes for providing training. For example, 
naval aviation maintenance personnel use of MTIP with 
refined CBT would produce better trained and more qualified 
technicians. This results in less time spent for 
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maintenance and improved quality in the maintenance 
performed. These factors will significantly contribute to 
higher sustained readiness for today’s modern sophisticated 
aircraft, support equipment and systems. 

D. SCOPE OF STUDY 

This research is applicable to related aspects of 
existing and future computer-based education, instruction 
and training. The study evaluates a limited military 
application for the use of CBT to further enhance MTIP. The 
training of enlisted naval aviation maintenance personnel is 
the target. Such an environment is characterized as having 
standardized procedures and uses easily verifiable data. 

The training is required for a large number of people and 
must be conducted frequently. A wide variety of students 
must finish training with a minimum level of skill and 
knowledge . 

Research is based on current books, periodicals and 
technical reports. Application is made to the military 
training environment. Recommendations based on the research 
are intended to support management policy-making to provide 
the best CBT for the military maintenance training 
environment with the resources available. 
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E. THESIS ORGANIZATION 



Chapter II provides a brief historical perspective of 
CBT. Examples are given of potential future applications 
for CBT. 

In Chapter III the fundamentals of database processing 
are described. Examples are given of how the Navy can and 
does use database management in CBT. The chapter concludes 
by enumerating several major advantages and disadvantages of 
database processing. 

Chapter IV explains features of Decision Support Systems 
(DSS). The attributes of the DSS major subsystems — data 
management, dialogue management and model management — are 
presented. Ways in which DSS can be applied to CBT and Navy 
training programs, especially MTIP, are described. 

The chapter also discusses Expert Systems used for 
training. The three major problems of using Expert Systems 
for knowledge acquisition are presented and a model of an 
Expert System used for training, TUTOR, is illustrated and 
described . 

Chapter V is about specific issues of CBT and the Navy’s 
Maintenance Training Improvement Program (MTIP). 
Instructional Systems Development is presented as the 
preferred method of CBT introduction for MTIP. 

Chapter VI follows with conclusions and recommendations 
about the impact and effectiveness of CBT on the MTIP and 
Navy training programs in general. 
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II. BACKGROUND 



A . HISTORICAL PERSPECTIVE 

Early developments in Computer-Based Training (CBT) were 
in the realm of Computer-Managed Instruction (CMI). CMI 
served as the foundation on which Computer-Aided Instruction 
(CAI) was built. CMI provided an environment which promoted 
and fostered CAI development as more cost efficient while 
providing more effective learning. 

Several reasons contribute to the CMI evolution as a 
foundation on which CAI developed to form the principle 
components of CBT. First, computer hardware was very 
expensive. Budgets did not provide funding for high start- 
up costs of computer equipment and facilities for CBT. In 
many situations it was not economically feasible to replace 
conventional instruction with CBT or to initiate CBT to meet 
an unfilled training need. 

Additionally, computers were difficult to use without 
extensive training. Most people could not readily use 
computers and their use was not conducive to the direct 
learning process. Computer literacy was low. Thirdly, the 
software for CBT was very limited in quantity and the 
quality needed to build dynamic learning systems. 

Presently, sophisticated software is readily available which 
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is much more supportive of the active, natural learning 
process for a large number of people. 

Several early systems were developed and have become 
well known. First among these is PLATO (Program Logic for 
Automatic Teaching Operations). It was originally developed 
at the University of Illinois and has now matured over two 
decades. It is currently marketed by Control Data 
Corporation. TICCIT (Time Shared Interactive Computer- 
Controlled Information Television) was developed by the 
Mitre Corporation and Brigham Young University. It was 
marketed by the Hazeltine Corporation beginning in 1976. 
Finally, AIS (Advanced Instructional System) is a system 
developed by McDonnell Douglas Corporation in conjunction 
with the Air Force Human Resources Laboratory. [Kearsley 
83:pp. 70-74] 

The U. S. Navy is regarded as a pioneer in early CBT 
research and development efforts [Hansen 75:p. 7]. These 
early efforts were in the area of CMI and later evolved and 
matured into the broader manifestations of CBT. In the 
1950’s and 1960’s, the Navy, through the Office of Naval 
Research (ONR) (as it was known then), sponsored and 
directed CMI research by various university and other 
organizational research centers. The Naval Training 
Research Laboratory (NTRL) (as it was known then) was 
established. CMI was initiated at the Naval Air Technical 
Training Center (NATTC), Memphis in Millington, Tennessee. 
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Although leading in development and research, the Navy was 
somewhat slower to implement early CBT than the Army or Air 
Force [Fletcher 86:pp. 183-197]. 

However, the Navy, as well as society as a whole, found 
itself surrounded by increasing technological 
sophistication. People were required to master more 
complicated subject matter and display more technical 
skills. Technology brought advances, i.e., improvements in 
job performance and conveniences; but it was less tolerant 
of ignorance or errors in judgment. Implementation of more 
advanced technological skills required higher standards of 
performance. CBT demonstrated a method of learning which 
reduced cost while increasing learning effectiveness. 

Technology increased productivity with a positive return 
on the investment of resources. Thus there was a 
"technology push" to advance and further improve the state 
of CBT. Through CBT technology, human capabilities are 
extended; they are "leveraged" to improve and expand human 
productivity and accomplishment. 

CBT differed from earlier learning methodologies 
predominantly in that it invoked active learning. A learner 
had to do something — answer a question, select a topic, ask 
for help, etc. Computer technology was combined with 
learning. It also provided the avenue for a person to learn 
interactively from a machine or a computer controlled 
system. Although training systems could be developed which 
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were passive, the most effective systems were those which 
were interactive. 

B. REASONS FOR COMPUTER-BASED TRAINING 

The rationale for CBT is fairly straightforward. It has 
long been recognized and accepted that no two people learn 
at the same rate, i.e., learning abilities vary 
considerably. Each person can learn most effectively when 
the sequence of instructional material, the pace and mode of 
instruction and even the style of presentation are tailored 
to his individual capabilities and desires. CBT offers an 
opportunity for highly individualized and selective training 
[ Lynch 7 1 : p . 4 ] . 

Almost all CBT applications are developed by the need to 
improve cost efficiency and learning effectiveness. While 
these are fairly general goals, they normally manifest 
themselves in more specific objectives [Kearsley 83:p. 2 
ff]. Some of these more specific reasons for CBT are 
illustrated in Table 2.1. These reasons are amplified 
individually and some potential future considerations are 
presented in the following pages. 

1 . Improved Control 

Training can be made uniform and established at a 
desired level of standardization which is optimal for the 
organizational needs. Consider the case of a maintenance 
training course where trainees must spend time practicing 
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TABLE 2.1 REASONS FOR COMPUTER-BASED TRAINING 
[ Rear sley 83 : p . 2 ] 



1. Improved Control 

2 . Reduced Resource Requirements 

3. Individualization 

4. Timeliness and Availability 

5. Efficient Use of Training Time 

6. Improved Quality of Training 

7. Discretionary Management of Training 

8. Change Agent 

9. Improved User Satisfaction 

10. Consistency 

11. Reduced Development Time 



repair of actual equipment malfunctions. Before students 
attempt the repairs, CBT can provide control to ensure that 
proper trouble-shooting techniques are taught and followed 
and that good diagnostic skills are developed. With a solid 
basis of theoretical and procedural training, trainees may 
further develop and refine their new skills with actual 
physical ’’hands on” training. The entire learning process 
is controlled and evaluation of learning accomplished is not 
based only on certain checkpoints along the way. The 
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process itself as well as the achievements are open to 
inspection and review. 

Computers also improve control by facilitating the 
administrative functions of the training system. These 
functions are in the realm of CMI. They may include 
registering students, assigning training schedules, 
generating and scoring tests, producing required reports, 
distributing instructional material and other necessary 
functions . 

CBT can be used to monitor student progress. 

Students receive instruction which is intended to provide at 
least a minimum level of comprehension and result in 
improved job performance above a standard level. However, 
comprehension rates vary from person to person and subject 
to subject. Improved control can ensure that these 
differences are quickly identified and action taken to 
provide optimum scheduling and assistance to promote student 
mastery of the subject material. 

2 . Reduced Resource Requirements 

An established CBT system can reduce resource 
requirements. However, significant, high start-up costs 
must be considered when implementing a new CBT system or 
converting from a conventional training program to CBT. 

These initial costs must be weighed against the life cycle 
benefits to be gained from CBT, including future rapid 
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assimilation of state-of-the-art training improvements which 
may be obtained from later CBT developments, 

CMI may reduce resource requirements by 
necessitating fewer people and facilities to maintain 
records and administer a training program. This reduces 
some of the overhead costs of training, CBT is 
individualized and tends to reduce training man-months. 
Therefore, fewer instructors, classrooms and instructional 
materials are required for a given student load compared to 
conventional training. 

CBT permits higher student-to-inst ructor ratios. 
Routine aspects of training may be accomplished more 
efficiently by CBT and instructors have more time to help 
students with problems. The CBT system can, in fact, 
pinpoint specific problems of each student for the attention 
of the instructor. 

Another important reduction in training resources 
from CBT is the use of computer-based simulators and 
trainers instead of actual equipment, such as aircraft, for 
training. Hands-on practice with actual equipment is 
critical to some training programs. If equipment is 
unavailable, crucial trouble-shooting or diagnostic training 
may be diluted. Computer-based simulators and trainers can 
provide realistic practice without the full cost of actual 
equipment. Finally, computer graphics can provide a student 
with a broader understanding of equipment operation than is 
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possible with mere practice on actual static equipment in 
the training environment. 

3 . Individualization 

Individualization has always been a principle 
objective of CBT because of the efficiencies it brings to 
the learning process. Each student is allowed to learn at 
his own pace and in the fashion most suited to him. This 
results in substantial time savings to complete training 
(See Table 2.2). In conventional training, instruction is 
often geared to a set pace based on the aggregation of past 
student performance. Thus training time for each student is 
the same even though student learning abilities may vary 
widely. Individualization, as provided by CBT, allows 
students to learn at their own pace and, thereby make better 
use of total training time. 

CBT systems also allow students some control over 
the sequence of instruction. While completing a training 
course, for example, students may spend more time with 
topics they find difficult to master and less time with 
topics they understand. CBT may also give students more 
discretion in scheduling their training time. 

Students begin training with different past 
experiences, levels of skill and knowledge. With CBT, 
student skill and knowledge levels may readily be assessed 
at the beginning of training and during execution. An 
individualized training program may be created which best 
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matches the needs and abilities of the student and the 
requirements of the organization for which the training is 
conducted. Maximum benefit may be obtained from a training 
opportunity. CBT can aid the training manager’s decision- 
making; more to come about that later. 

Individualization is important for remedial 
instruction conducted during or after training. The Navy 
frequently uses remedial training in schools and as part 

of the Maintenance Training Improvement Program (MTIP). CBT 
can identify specific problem areas and provide assistance 
to ensure that each student completes training with an 
acceptable level of skill and competence. 

4 . Timeliness and Availability 

CBT provides significant advantages over 
conventional classroom training by being able to provide 
training ”on demand.” Training can be provided at the right 
time and in the right place. Students need not wait for 
specific training to become available. Pertinent training 
can be made available exactly as required. 

Students typically are provided certain training 
periods when they are absent from their normal jobs. During 
these periods they may be given an entire syllabus of 
training for immediate and future needs. Unfortunately, 
much of the training for future needs is often forgotten or 
becomes obsolete by the time it is needed. CBT allows 
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training to be provided in smaller, more directly applicable 
packages . 

CBT has the potential to provide training at remote 
sites. The training can come to the students instead of the 
students going to the training. Much of the travel time and 
expense for training can be eliminated. The training can be 
dispersed by providing instructional material to each 
requiring site or by distribution over a computer network 
from a central facility. 

Finally, students are able to learn about new 
equipment before it has arrived through CBT trainers or 
simulators. Thus they are more proficient in operating the 
new equipment when it does arrive. Students may also be 
trained in new maintenance procedures and techniques as soon 
as they are introduced. 

5 . Efficient Use of Training Time 

With CBT, trainees can obtain the same level of 
learning in less total time than with conventional 
instruction. However, CBT is not a training panacea. In 
the military setting, where large numbers of people need 
training frequently, it may be very appropriate [Dean 83:p. 
150]. For example, a study of the effectiveness of CBT in 
the military demonstrated time savings of about 30 percent 
based on findings for Air Force and Navy training [IDA P- 
1375, 79:pp. 5, 83]. See also Table 2.2. 
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More efficient use of training time is almost 
completely attributable to individualization. Thus non- 
computerized training could also be made more time efficient 
with more individualization but probably at higher cost. 
However, as a practical matter, it is virtually impossible 
to establish and manage a large individualized training 
program without the use of a computer. 

With a CBT system, an assessment of student skill 
and knowledge levels may be facilitated prior to and during 
a training course. Then a course of instruction may be 
promptly created and modified along the way to provide the 
student with the most learning potential for the training 
time available. The student’s time is not wasted going over 
subjects he already knows and understands. Rather, he is 
challenged with unknown material and can make better use of 
the training time. 

More efficient use of training time permits students 
to train but spend less time away from their normal jobs. 
Sometimes, the length of training is the criterion for 
determining if that training can be taken or not. CBT has 
the potential, under favorable conditions, to provide the 
most training for a specified time period or provide 
specified training in the minimum time. 

The efficiencies of CBT may require changes to an 
existing training system and organization. Conventional 
training systems built around fixed schedules must change to 
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take advantage of CBT. For example, when the Navy first 
introduced CBT, training assignments remained on a fixed 
schedule. Most students finished their training early and 
were assigned undesirable tasks, i.e., watchstanding, 
recruiting duty, mess cooking or clean-up detail, while 
waiting for their scheduled training time to end. The word 
quickly spread that finishing training early was not a good 
idea and training completion time actually increased to 
nearly match the scheduled time. The Navy soon changed the 
assignment procedure from fixed duration to a continuous 
method and the problem disappeared. [Kearsley 83:p. 10] 

6 . Improved Quality of Training 

Quality of training is difficult to measure and 
prone to subjective evaluation. Criteria may be established 
by which to appraise quality and measure effectiveness. The 
measurements may be direct or indirect. Direct measurements 
are those which gauge training based on evaluation of the 
training process itself and not its effectiveness. Indirect 
measurements are those which gauge training quality based on 
evaluation of the results of training. Both direct and 
indirect assessments may be important. Direct results 
provide immediate indications of discernible deficiencies 
which are usually readily correctable. 

The ultimate measure of the quality of training is, 
however, its effect on improving subsequent job performance, 
an indirect and subjective measure of training quality. The 
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purpose of training must be to improve job performance and 
that must be held at the forefront of all training programs. 
The users of CBT must evaluate its impact on improving job 
performance. High quality training should translate into 
improved subsequent performance in an existing job or lead 
to high quality initial job performance. 

Orlansky and String have compiled approximately 30 
studies (48 sets of data) which report on the effectiveness 
of CAI and CMI in military training. These results are 
presented in Table 2.2. 

CBT provides several features which tend to improve 
the quality of training. In interactive, individualized 
instruction (such as good CBT), students spend a much higher 
percentage of time involved in the process and learning, 
i.e., students participate more in training than in 
conventional instruction. Secondly, CBT can provide 
pertinent feedback to trainees and instructors. Feedback 
may reinforce positive accomplishment in the form of 
messages and new assignments. Instructors may also 
intervene with praise, recognition or provide special 
privileges. Feedback may also suggest corrective action for 
less than satisfactory progress thus ensuring that training 
meets at least the minimum of established quality standards. 

CBT may be used to improve job performance directly 
or indirectly. In the direct approach, students are trained 
in specific skills for their immediate job situation. An 
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TABLE 2.2 SUMMARY OF FINDINGS FOR MILITARY CAI AND CMI, COMPARED TO CONVENTIONAL INSTRUCTION 

[IDA P-1375 79: p. 84] 
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example is inservice training such as the Navy’s MTIP. Job 
performance is expected to improve as students are trained 
in specific skills for their immediate job situation. Job 
performance is expected to improve for a particular job 
role. Using an indirect approach, instruction may be 
provided which is more general purpose and more broadly 
affects job performance, such as remedial instruction in 
mathematics, reading or finance. This training may be 
helpful for many jobs and in the student’s personal life as 
well. 

7 . Discretionary Management of Training 

CBT may be more accommodating and flexible than 
conventional training to trainees and employers of trainees. 
These characteristics may lead to improved morale and 
productivity . 

Students have, within limits, the freedom to pursue 
training at their own pace and to fit their own preferred 
schedule. More time may be spent on topics which are more 
interesting to the student or which require more time to 
master. The trainee may receive feedback to indicate areas 
requiring more study or to confirm perceived levels of 
training comprehension. 

Manager s~-those who supervise training programs and 
those who send t rainees--ar e afforded more professional 
discretion in their positions. A training course can be 
started at any opportune time. It is not imperative that a 
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class schedule be set and then quotas filled with students 
for training to be conducted. Courses of instruction can be 
easily modified or updated as valid needs arise with good 
database management as discussed later. Additionally, CBT 
provides the technology to provide training to the user 
sites, if warranted, rather than requiring that training be 
conducted at a central training facility. Courses may be 
readily tailored to the needs of individual customers. 

With computer technology becoming ever more 
ubiquitous and widespread, CBT provides the opportunity for 
students to work with and learn on a computer system. As 
they learn, they also become familiar with a computer 
system. They may even be able to learn on a system which is 
identical to or similar to one they will use in their job, 

8 . Change Agent 

CBT is a different way of presenting training than 
conventional classroom instruction. It is an alternative 
way to accomplish common or more ambitious goals, CBT can 
provide many benefits as described here. Incorporation of 
CBT in an organization will require managerial change as 
well as technological change to realize its maximum benefit 
[Kearsley 85:p, 61 ff, and 83:p, 13-14, Seidel 81:p. 211 
ff]. For example, application of a computerized Database 
Management System (DBMS) or Decision Support System (DSS) 
makes more sense with CBT than with conventional 
instruction. More about this in later chapters. 
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Note that change is made for organizational benefit 
and not merely for the sake of change. Change is made to 
provide a suitable environment for the new training system 
and an atmosphere in which it can function best. 
Effectiveness measures must be examined and re-evaluated. 
Instructor roles and functions must be made compatible with 
CBT. Users of CBT must learn how to use a computer. These 
are some of the most apparent changes driven by introduction 
of CBT. 

CBT allows changes in instructional materials and 
methods to be made quickly. There is no need to print and 
physically distribute changes to all concerned parties. 
Changes can be made in the computer system and are rapidly 
available to all users. If common CBT is conducted at more 
than one site, changes can be distributed and incorporated 
electronically. Potential system modifications may be 
tested in CBT by pilot, phased or prototype projects before 
being approved for system wide use. 

Because CBT is very accommodating to change, it 
promotes change. The wide variety of training 
presentations, i.e., text, graphics, trainers, simulators, 
encourages developers to create potential improved methods, 
material and equipment to deliver training. As these become 
proven and accepted, they can be implemented as constructive 
changes to a CBT system. 
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9 . Improved User Satisfaction 



Instructors may find improved professional 
satisfaction in CBT because the system facilitates many of 
the administrative and routine functions required of them. 
More of their time may be spent helping students and 
monitoring student progress. It is also easier to keep 
training material current and pertinent. When compared to 
conventional instruction, CBT offers high potential to 
provide many more training resources to instructors. 

In the training environment, student aptitude and 
attitude are important considerations for successful 
learning. CBT can stimulate positive attitudes. Student 
participation in the learning is high, leading to higher 
self-satisfaction with learning accomplishment. Students 
can somewhat tailor the sequence and pace of training to 
their personal style and are more involved in the learning 
process. [Kontos 84:p. 4] 

CBT, especially graphics, trainers and simulators, 
can provide more realism to training compared to 
conventional classroom training, thus giving the student 
more of a sense of purpose and achievement. Rapid response 
time and feedback are CBT characteristics which improve user 
satisfaction with the system. Additionally, students can 
assess their progress during training and that creates a 
sense of accomplishment and satisfaction. Improved 
satisfaction leads to increased motivation which strongly 
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complements any level of trainee aptitude. Together these 
tend to improve training productivity and reduce (possibly 
eliminate) training course failures. 

1 0 . Consistency 

Human instructors can have an ”off day’’ or miss a 
day because of illness or personal reasons. This interrupts 
a course of instruction. A substitute instructor will most 
likely not present the same material in the same manner as 
the originally scheduled instructor would have. With CBT 
more consistent training can be presented to trainees. 

During development, subject matter experts (SME) 
and training designers may work together to create material 
to cover a topic to meet course objectives. The consensus 
of these expert opinions can result in a computer-based 
course of instruction that will satisfy course objectives 
each and every time it is presented. 

1 1 . Reduced Development Time 

Training materials often require modification as 
procedures, regulations or equipment change. In fact, out- 
of-date materials is a major cause of ineffective training 
and dissatisfaction with training. Authoring can be 
facilitated with CBT. Also, making revisions to existing 
material based on new developments and implementing them is, 
frequently, time-consuming and difficult. CBT streamlines 
the revision process. 
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12. Future Enhancements 



Computer hardware costs have decreased dramatically 
while capability has increased greatly. If future advances 
are to be realized, CBT must be established as a foundation 
on which to build. Two most likely future advances, which 
may already be found in very limited use, are discussed in 
the following paragraphs. 

a. Embedded Training 

Embedded training refers to integrating 
training functions directly into weapon, aircraft or 
maintenance equipment systems. Embedded training is a facet 
of CBT which is developed with the corresponding equipment 
or system as an integral part of it. An advantage of 
embedded training is that it is very specialized training in 
which a person is taught on and by the actual equipment or 
system he will be using. Secondly, the training should be 
of high quality since it is produced by the same source as 
the equipment or system and designed specifically for that 
equipment or system. Also, training may be conducted in the 
field and is not limited to training facilities; it is 
available wherever the equipment or system is located. 

b. Networks 

Communication networks will permit training to 
be readily distributed to many users. Small users, who 
would not normally be candidates for a CBT system, may have 
access to CBT via a network. Modifications to existing 
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training material and new training material may be easily 
incorporated into the CBT system with virtually no delay and 
are quickly available to users. 

Networks allow for greater physical security by 
distributing duplicate programs and databases at various 
locations in the network. This may also permit the CBT 
system to function even though part of it is down for 
scheduled maintenance or other unplanned reasons. 

C. DISADVANTAGES OF COMPUTER-BASED TRAINING 

A major disadvantage of CBT is high start-up costs. 

These are generally one time costs for system development, 
facilities, hardware and software. As development is 
completed, equipment is procured, installed and tested. 
Additionally, the recurring or maintenance costs for the 
system must be considered for the life cycle. The benefits 
to be gained from a CBT system must be weighed against these 
costs . 

Rapid obsolescence may be another significant 
disadvantage. Computers, as instructional tools, appear to 
be at a rudimentary state of development [Bork 87:pp. 201- 
206; NPRDC TR 84-54, 84:p. 1]. Breakthroughs or rapid 
development in computer technology or CBT may make existing 
equipment or systems impractical or obsolete in terms of 
training effectiveness before their costs are recovered. 
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Unfortunately, computers can be manipulated or ’’gamed" 
by students. Part of this phenomenon has been formalized as 
Parkinson’s Law which states, ’’Work expands so as to fill 
the time available for its completion’’ [Parkinson 57:p. 1]. 

Another disadvantage is that while student attitudes 
generally appear favorable toward CBT, that of instructors 
has been unfavorable in some cases (See Table 2.2). 
Contributing to this is the fact that instructor roles have 
not always been clearly defined nor expectations described 
for CBT [NPRDC TR 82-45, 82;pp. 24-25]. 

CBT may also limit constructive interaction among 
students as they learn. As students learn individually, 
they may not advance and refine interpersonal skills which 
may be needed to work in more cooperative team roles for 
future jobs. 

D. SUMMARY 

The chapter has presented a brief history of CBT 
including some examples of early systems. Mentioned are 
some of the conditions and requirements which promoted CBT. 
Several reasons are presented for using CBT; these reasons 
generally contribute to training cost reductions and/or 
improved learning effectiveness. Also, some additional 
potential future uses of CBT were suggested. Finally, some 
of the major disadvantages of CBT, such as high initial and 
maintenance costs, were described. 
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The Maintenance Training Improvement Program (MTIP) is a 
CBT system and embodies several of the characteristics given 
for CBT. CBT can allow MTIP to be further improved and 
provide a foundation for future developments and 
enhancements as discussed later. 
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III. DATABASE MANAGEMENT APPLIED 
TO COMPUTER-BASED TRAINING 



Data is a very important resource in a Computer-Based 
Training (CBT) system. Data includes facts and figures 
about reality from which information may be drawn and 
conclusions made. Information is selected data about 
specific circumstances or subjects which convey new meaning 
to a receiver. Knowledge is general understanding based on 
experience and education or training. Knowledge, either 
personal or expert, is applied to applicable information for 
decision making. [Harmon 85:pp. 30-31, Kroenke 87:p. 41, 
Wiederhold 86:pp. 78-79] The data held by a CBT system is 
fundamentally important to trainees, instructors and 
managers. The foundation on which CBT is built and 
functions is provided by data. 

A CBT system must be able to receive or gather data. 
There may be many sources of data. For example, data for 
training may be obtained from equipment manufacturers or 
operators, or from other military organizations in the form 
of specifications or procedures. Data may be generated 
internally within CBT as the system is used. Users of CBT 
usually must provide some data to the system to be able to 
use the system. Other data stores, books, manuals, etc., 
may provide data to a CBT system. 
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Data which is input into the system must be subject to 
verification and periodic checks made to ensure the accuracy 
of all data. Once data is received and entered into the 
system, it must be capable of being accessed and processed 
in a reasonable time. Finally, data in a CBT system must be 
secure. Data should not be lost or easily stolen. 

Protection from unauthorized tampering, altering or copying 
must be provided. One common method of ensuring integrity 
is through data duplication. The capability to 
add, edit or delete data must be restricted to a few trained 
and authorized responsible individuals. 

Data and the subsequent information it provides are 
important to CBT users in slightly different ways. Students 
rely on data to provide information for training and 
learning. Thus, the student’s knowledge base is expanded 
which should promote improved decision-making for better 
future job performance. Data is converted into information 
and incorporated into student knowledge before it is needed. 
The data is used for decision making outside CBT. Feedback 
may point to changes necessary in the data to provide better 
training. This student-data relationship is illustrated in 
Figure 3.1. 

Managers of CBT systems depend on data in a slightly 
different manner. For them data provides information 
directly for decision making about CBT itself. Information 
supplements existing knowledge in a more immediate situation 
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Figure 3.1 Student-Data Relationship 
[Wiederhold 86;p. 78] 

and through feedback becomes part of experience. The 
manager-data relationship is illustrated in Figure 3.2. In 
the next section, a description is provided of how CBT 
provides a Management Information System (MIS) for the 
management of CBT. CMI is the component of CBT which 
includes the MIS functions. Figure 3.3 illustrates how data 
and databases fit into the scheme of a CMI subsystem. They 
serve a very central and crucial role. 
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Figure 3.2 Manager-Data Relationship 
[Wiederhold 86:p. 78] 



A. MANAGEMENT INFORMATION SYSTEM 
1 . Definition 

Management Information System (MIS) means different 
things to different people. At one extreme are those who 
view MIS as essentially equivalent to Electronic Data 
Processing (EDP). Thus data transactions are processed into 
scheduled reports. At the other extreme are those who 
believe that MIS should provide all the information which an 
organization needs to function effectively. Typically, much 
emphasis is placed on supplying management with information 
for decision making. [Sprague 77:p. 7] 
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Figure 3.3 Typical Components of a CMI Subsystem 

[Kearsley 83:p. 25] 



Gorry and Scott Morton [Gorry 71:pp. 69-79] describe 
a MIS framework for managerial activities. Managerial 
activities are categorized as strategic planning, management 
control and operational control. Strategic planning is the 
process of establishing objectives for an organization. 
Through management control, managers use judgment to assure 
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that resources are obtained and implemented effectively and 
efficiently to accomplish the organization’s objectives. 

The third general function of operational control assures 
that system tasks are accomplished expediently and 
economically by carefully and previously delineated 
procedures . 

Decisions which are required of managers may also be 
classified as belonging to a continuum from structured or 
unstructured. Structured decisions are usually repetitive, 
routine and follow set procedures. Unstructured decisions 
are novel, complex, occur infrequently and follow no set 
procedures. Figure 3.4 illustrates the areas of information 
systems generally thought of as MIS. The Structured 
Decision Systems (SDS) encompass nearly all of what has 
traditionally been called MIS. 

Gorry, Scott Morton and Sprague [Sprague 80:p. 10] 

reject the traditional definition of MIS for a broader, more 
practical view in which MIS refers to the whole set of 
systems and activities necessary to gather, process, manage 
and ultimately use data and information as a resource in an 
organization. The area of Decision Support Systems (DSS) 
applied to CBT, will be discussed in the next chapter. 

2 . Database Management as a Foundation for MIS 

Data from which to derive information is essential 
to MIS. This data consists of data elements that comprise 
an organization’s database. In the late 1960’s, Head [Head 
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Figure 3.4 SDS and DSS in Management Information Systems 

[Gorry 71:p. 74] 

67:p. 23] explained that to accomplish anything useful with 
the database, it is necessary to provide organization and 
structure. To do this, the information requirements of 
management must be identified. The data elements must be 
identified in technical terms: how large they are, what 

their meaning is, where they are stored and how one can get 
at them. Finally, data relationships among data elements 
have to be identified. 

Differing levels and types of management require 
different types of information (see Figure 3.3). This can 
result in many specialized databases. In CBT, for example, 
instructors, training program managers and the Chief of 
Naval Education and Training (CNET) could each have their 
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own specialized databases. This hierarchical or 
"horizontal” design matches the database with the 
organizational level which is its primary user. Such a 
database is likely of very limited or no use to others in 
the organization. A proliferation of specialized databases 
may result in much duplication of an organization’s core 
data. Databases have also been designed functionally or 
"vertically" to serve all levels of management with data 
about a particular subject. For example, in CBT, separate 
databases could be maintained for student records, test bank, 
learning activities (see Figure 3.3) and aircraft 
maintenance data for the Maintenance Training Improvement 
Program (MTIP) . 

Two particular advances in technology have aided MIS 
development. These are the development of large data 
storage devices and interactive, real-time computer systems 
such as Database Management Systems (DBMS). The DBMS can 
produce a more responsive MIS and provide all levels of 
management with the functional information required from a 
common organizational database. The DBMS gives the 
organizational database design and structure, establishing a 
foundation for MIS as illustrated in Figure 3.5. The DBMS 
provides maintenance of the database as well as a 
communication network for gathering and modifying data. It 
also provides for inquiry to the database for flexible 
access and information retrieval [Sprague 77:pp. 8-9]. 
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Figure 3.5 Organizational Model for MIS 
[Head 67:p. 23] 



B. COMPONENTS OF A DATABASE MANAGEMENT SYSTEM 

CBT employs an MIS which involves multiple users, 
multiple data files and requires multiple views of stored 
data. A Database Management System (DBMS) is a system of 
components which provides a more sophisticated approach to 
data management, storage and retrieval than traditional file 
systems. The five major components of a DBMS are considered 
applicable to CBT; they are: hardware, programs, data, 

people and procedures. 

1 • Hardware 

With a computer installation such as CBT in place, 
there is very likely no need for additional specialized 
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hardware to implement database processing. A DBMS may, 
however, require more memory and storage components to 
accommodate its increased overhead. Several characteristics 
of hardware which would be advantageous to CBT would also 
facilitate DBMS. More main memory, faster Central 
Processing Unit(s) (CPU) and more direct access storage may 
be essential for the DBMS to function effectively. 

Recently specialized database machines have been 
introduced by manufacturers. These machines are special 
purpose computers whose only function is to manage the 
database. (See [Date 86:p. 603 ff] for a description of 
these machines.) Each CBT system would have to be analyzed 
independently to determine if such a machine could be used 
effectively in that system. 

2 . Software 

Software refers to the programs used in the database 
processing system. Database requests and transactions may 
be processed in batch form or on-line by users at terminals. 
On-line processing uses a Communications Control Program 
(CCP) to receive and route requests. CCP provides 
communication error checking and correction. It coordinates 
activity among the various terminals and routes messages 
within the system. 

Other types of programs are application programs and 
utilities. Application programs satisfy specific 
predetermined needs such as class lists, tests, learning 
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activities, etc. They must be developed for or tailored to 
each CBT system. Utility programs provide a wide variety of 
services. They may provide for generalized update and query 
of the database. Software locks are included to provide 
security for classified data. Others establish and maintain 
the database and can reformat and clean-up files for more 
effective operation and more efficient storage. 

A third type of program is the Operating System. 

This program controls other programs and controls the 
computer's resources. The operating system is usually 
designed for specific hardware and provided with it. The 
DBMS shields the user from any direct contact with the 
operating system. Figure 3.6 illustrates the programs 
involved in typical database processing. 

3 . People 

People are an important part of the CBT DBMS. 
Obviously, there are users who employ the system every day 
for its intended function. As in other computer-based 
systems, there are operational and systems development 
personnel who are included in the DBMS. 

A very prominent position in the DBMS is that of the 
Database Administrator (DBA). Often referred to as a single 
person, this may actually be a group of people. Database 
administration is key to providing an environment for 
effective DBMS employment. A function of the DBA is to 
manage data activity. The processing rights of all users 
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Figure 3.6 Programs Involved in Typical Database Processing 

[ Kroenke 83 : p . 9 ] 



must be determined. The DBA is the focal point for 
negotiating and assigning these rights. Different users may 
have different priorities. In a CBT system, for example, 
students and instructors are likely the most frequent users 
and have very high priority in keeping with the mission of 
training . 

However, required reports or management activity 
may occasionally pre-empt student and instructor use of the 
system. Tasks which are not mission critical should be 
scheduled for off-peak hours. Once these access rights are 
originally determined or later modified, they must be 
documented and distributed to applicable users. The DBA 
must also formulate additional operating procedures for such 
items as concurrent database update, backup of the existing 
database and recovery of the database in the event of system 
failure. A CBT application where the role of the DBA is 
very important is the Navy Maintenance Training Improvement 
Program (MTIP) which consists of many databases each 
requiring sound, responsive management. 

The DBA is a central figure in the management of the 
DBMS itself. This is discussed further in the next section. 
Finally, the DBA is concerned with DBMS performance. A DBMS 
can be periodically tuned for more efficient operation in 
its present environment. This makes better use of existing 
resources to achieve system objectives. Evaluating current 
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performance helps determine which new software or equipment 
to procure for future requirements. [Kroenke 87:pp. 324-327] 
4 . Data 

As stated earlier in the chapter, data is a very 
important resource for CBT as a foundation for the DBMS. 

Data in a database is integrated and shared. The database 
must be self-describing, i.e., it contains a description of 
the database structure. A self-describing database ensures 
that the data remains application program independent. A 
DBMS should provide a data dictionary, or user-accessible 
catalog which describes the data [Kroenke 87:p. 317]. 

An integrated database is more than a collection of 
data files. It is a collection of files with much 
redundancy eliminated, but includes designed relationships 
among the various file records. A shared database is one 
where different users, through the various application 
programs, have access to the same pieces of data. 

Data may be referred to in three additional ways in 
a database system. First is the logical view of all data as 
it would appear to humans in general. This is also known as 
the schema or conceptual view. Another view is the external 
or subschema perspective. This is a particular view by a 
specified user and provides different views of the same data 
depending on the particular perspective. There can be many 
overlapping subschemas for a given database and each is a 
subset of the general logical view. A third view is the 
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internal or physical view of data as it is actually re- 
presented. Figure 3.7 illustrates the three views of data. 

To implement the logical database, physical 
considerations are imposed on the conceptual model to create 
the physical database [Cesena 87:p. 5]. This is the view of 
data as it is stored and processed in a particular system* 
Application programs call the logical files which are stored 
in the physical database. The various views and 
relationships of data logical files as used by application 
programs in the CBT physical database is summarized in 
Figure 3.8. [Date 86:pp. 5-7, Kroenke 83:pp. 11-14] 

5 . Procedures 

DBMS procedures are both system operating 
instructions and policies for system use. Procedures are 
established for system users and operators. Procedures may 
also be segregated as those for normal conditions or those 
for contingencies such as emergency, failure or recovery 
occurrences. As noted above, development of system 
procedures is a prime responsibility of the DBA. When 
establishing procedures, the requirements of all users 
should be considered with a view to the technical, 
operational and economic feasibility of each [Davis 83:pp. 
38-39]. Compromises or alternative solutions must be sought 
when conflicts arise. Procedures should not be at cross- 
purposes with one another and should promote optimal use of 
system resources for the benefit of all users. 
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Figure 3.7 Conceptual, External and Internal Views of Data 

[ Cesena 87 : p . 5 ] 
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Figure 3.8 Physical Representation of Logical Files 
in an Integrated and Shared Database 
[Date 86:p. 6] 
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C. MANAGEMENT 



As its name implies a DBMS is a management tool. The 
DBMS is very important for a MIS and has several components 
as discussed previously. Kroenke [Kroenke 83:p. 538 ff] 
describes three aspects of management associated with a 
DBMS. The first two, management of data base structure and 
management of database activity are implemented by the DBMS. 
The third is the management of the DBMS itself. Each of 
these has application for the CBT system. 

1 . Management of Database Structure 

For database processing to function properly some 
standardization must exist. This provides a common 
reference for all users. A data dictionary which contains 
the names and structures of all records and files is helpful 
in this regard. Communication between users and DBMS and 
between users themselves will be improved. 

Database systems tend to reduce or eliminate data 
redundancy. However, some duplication may be required or 
desired to improve system performance or security. 

Redundancy which is permitted must be managed to ensure that 
recurring data is periodically reconciled. 

DBMS management must provide for configuration 
control. Database standards must be enforced. However, a 
process must be designed for making constructive changes as 
needed to accommodate a changing DBMS environment. As their 
requirements change, users should be permitted to suggest 
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appropriate changes to the structure. Information about the 
DBMS structure and changes made to it must be documented and 
distributed to users. 



2 . Management of Database Activity 

Data ownership is an important part of database 
activity. Who owns which data? How is data entered, 
accessed or modified? Who is responsible for the accuracy 
of data? Management must deal with each of these concerns. 

Security must be provided for any classified data 
which may exist. Additionally, physical security must be 
provided against theft, loss, fire, etc. Recovery 
techniques and procedures must be established in 
anticipation of machine malfunctions, communication failures 
or incorrect system use. 

Database activity must include training for new and 
existing users. Initially, new users are to be provided 
with training to use the system correctly. As the system 
changes, procedures and policies must be modified and 
existing users informed and retrained if necessary. 
Management must enforce data activity policy and procedures 
for the benefit of all users. 

3 . Management of the DBMS 

Effective operation of a DBMS must be planned and 
controlled. In addition to periodic tuning of the system, 
provision must be made for maintenance and modification. 
System use and performance should be evaluated as to how 
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well user requirements are being met. New products and 
services from industry and advances in technology should be 
monitored for application to the system to more economically 
meet user requirements, 

D. ADVANTAGES OF DATABASE PROCESSING 

A major advantage of database processing is that it can 
provide a CBT system with centralized control of its 
operational data. Centralized control allows more 
information from the same amount of data compared to a 
typical file system. Data may be more efficiently shared 
among legitimate users. With database processing, data 
redundancy may be significantly reduced and data 
inconsistency avoided thus providing more accurate data. 
Finally, database standards may be established and applied 
to an entire organization. With a database, one department 
or group can better manage the data through specialization 
and by providing a central point to implement database 
policies and procedures. 

E. DISADVANTAGES OF DATABASE PROCESSING 

Database processing has several potential disadvantages. 
First and foremost is the fact that it can be complex and 
expensive. High expenses result from the initial cost and 
additional hardware needed to support the system. 

Complexity implies high operating overhead which will 
actually slow down some simple sequential processes. A 
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complex system is more subject to degradation and failure. 

A system failure may affect most or all contained processes. 
With a large and complex system, recovery from failure is 
more difficult. It is also more difficult to provide 
security for a large and complex system which has many 
users . 

F. SUMMARY 

The chapter began with a discussion of data as a CRT 
resource, which can provide information and knowledge. As a 
resource, data is of critical importance to CRT in general 
and the Navy’s Maintenance Training Improvement Program 
(MTIP) as discussed later. CRT students, instructors and 
managers use data to achieve their specific purposes. Data 
elements or the database serve as a foundation for the CRT 
Management Information System. Data is a critical resource 
of CRT and a DRMS offers the potential to manage it better. 
Managerial activities are classified as strategic planning, 
management control and operational control. Decisions 
required of training managers lie on a continuum between 
structured and unstructured. To meet these varying 
managerial requirements, specialized databases can be 
designed, which are often useful to very few users. DRMS 
permits necessary information to be obtained from existing 
databases and can provide a more responsive MIS for all 
levels of management and operational users. 
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The chapter concluded with a discussion of the major 
components of a DBMS and their functions. The success of 
CBT rests in large measure on the ability to manage and 
effectively use data. Database management was described as 
management of data activity, data structure and the DBMS. 
Finally, several advantages and disadvantages of database 
processing were presented. 
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IV. INTEGRATION OF DECISION SYSTEMS AND 
COMPUTER-BASED TRAINING 



A. DECISION SYSTEMS 

The decision-oriented approach to information systems 
attempts to identify the full range of alternatives and 
resources available to a decision-maker. Two types of 
decision systems will be considered as applicable to 
Computer-Based Training (CBT). The first is a Decision 
Support System (DSS) which supports decisions. The second 
is an Expert System which makes decisions. These systems 
may be integrated for certain applications or employed 
independently. [Turban 85:p. 141] In either case, the 
decision action to be taken is usually left to the decision- 
maker . 

Mason [Mason 69:p. 81] directs attention to an important 
question. Namely, what is the optimal point of articulation 
or juncture between an information system and the decision- 
maker? That is, what are the roles of the information 
system and the decision-maker? Where does the information 
system begin and leave off and the decision-maker take over? 

Figure 4.1 illustrates three possible points of 
articulation which are represented by Database Management 
System (DBMS), Decision Support System (DSS) and Expert 
System. In Figure 4.1, decision-making is presented as a 



55 



sequence of events culminating in a course of action based 
on a decision being made. The sequence of activities is 
summarized as follows; 

1. Source — origin of data 

2. Data — used to provide information 

3. Inferences and Predict ions--based on information 

4. Value and Choice--apply subjective criteria and 

ratings 

5. Action--make decision and proceed with course of 

action 

Identifying and defining these activities is key to 
designing an information system. 

Figure 4.1a represents a DBMS which is described in 
Chapter III. The information system receives, classifies 
and stores data from which a decision-maker may draw 
information. The information system is designed to contain 
all the data necessary to provide information for the 
pertinent decision-making application. The database 
structure emphasizes economy with data duplication, 
inconsistency and omission minimized and superfluous data 
purged regularly to meet system objectives. This 
configuration presents facts and the decision-maker is left 
to determine "meaning." This design is best suited for 
managerial control situations where decisions are mostly 
objective and structured. Policies and procedures tend to 
be explicit and well established administratively. In CBT, 
it is this part of the system which provides scheduled 
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Figure 4.1 Information System and Decision-maker 
Relationship [Mason 69:pp. 82-86] 
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reports such as class lists, student schedules and other 
routine reports. 

An information system may extend beyond the activities 
of pure data collection, storage and retrieval to include 
the drawing of inferences and making predictions which are 
relevant for decision-making. This type of information 
system is a DSS, illustrated in Figure 4.1b. This process 
assumes certain relationships which are built into the 
system. An example is simulation models. The decision- 
maker is permitted to inquire "what if?" certain actions are 
taken based on assumptions which are considered valid. The 
system then provides results for all desired inputs based on 
the simulation model. The system does not evaluate outcomes 
and the decision-maker is left to assess outcomes and choose 
a preferred course of action. The next section provides a 
discussion of the major components of a DSS. An example 
from CBT, is the evaluation of an increase in course 
content. How does the change affect student schedules? Are 
the required resources available when needed? If course 
prerequisites change how will student loading change? 

A further evolution of the information system is an 
Expert System as pictured in Figure 4.1c. Such a system 
incorporates an organization’s value system and criteria for 
choice. The design can be referred to as a decision-making 
system. The system will make a decision and the decision- 
maker then has an opportunity to reject the decision, accept 
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as is, or accept with modification and then begin a course 
of action. As its name implies, this process attempts to 
mimic the behavior of experts. The system incorporates the 
algorithms of experts to provide the best or optimal 
solution in a particular area of expertise. Expert systems 
are discussed in more detail later in Chapter V. 

As the information system develops through the phases 
discussed above, assumptions become intrinsic to the system. 
These assumptions may become ’’hidden” to the decision-maker. 
The underlying assumptions must, however, be in accord with 
the best judgment of the decision-maker for the information 
system to become a valuable management tool. Assumptions 
must be accurate and regularly updated. The decision-maker 
must develop confidence in the system for it to be employed 
effectively . 

B. CAPABILITIES OF A DECISION SUPPORT SYSTEM 

DSS is not merely an evolutionary advancement of EDP and 
DBMS. It is a class of information system which draws on 
EDP and DBMS but certainly will not replace either. DSS 
requires a new combination of information systems technology 
to satisfy new requirements. The objective is to further 
improve the decision-making process. [Sprague 80:pp. 10-12] 
In designing a DSS, the components of the system must be 
identified and their interface specified. Three prime 
capabilities are crucial in a DSS. These capabilities are 
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provided by the data management, dialogue management and 
model management subsystems. [Sprague 80:p. 29] 

1 . Data Management 

A collection of data maintained by a training center 
for planning, control, training and administration is its 
database (see Chapter III for further discussion). Sprague 
and Carlson [Sprague 82:pp. 223-225] suggest that database 
management is important to DSS for two reasons. First, a 
database and associated DBMS are needed as a prerequisite to 
building a DSS because building a DSS without them would be 
very difficult. They are basic building blocks for a DSS. 
The reasons for having a database prior to building a DSS 
may be summarized as follows: 

1. Simplifies collection and maintenance of data used 
by a DSS 

2. Limits the functions and users that a DSS must 
support 

3. Simplifies DSS design 

4. Reduces potential conflicting performance and 
security requirements 

5. Increases the opportunity for sharing data within 
a DSS while reducing data redundancy 

Having a database prior to building a DSS can reduce the 

cost of building a DSS while improving the accuracy of the 

data used in a DSS. 

Secondly, data management is a major capability 
required of a DSS. Data management supports the memory 
requirements of a DSS. Data for DSS use is stored in 
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databases. Data management provides workspace for 
intermediate results and provides libraries for saving 
workspaces. Data management links related data, reminds 
decision-makers of operations to be performed and of data 
that should be considered. 

Data management includes the ability to reduce or 
abstract desired information from large amounts of data. 

For example, in evaluating a course of instruction or a 
training session, a training manager may want to review the 
prior training of a group of students who completed the 
course in question. With data reduction, he should be able 
to obtain the required information without going through 
each student record. 

DSS should provide varying levels of detail. 

A manager on the staff of the Chief of Naval Education and 
Training (CNET) may need information about how Aviation 
Machinist’s Mate (AD) students have been graded over the 
past year. On the other hand, an instructor may wish to 
know how a particular class of AD’s has done on tests during 
an ongoing course. 

2 . Dialogue Management 

Dialogue is the interface between the decision-maker 
and the DSS. It is interactive and determines the user 
friendliness of a DSS. Dialogue management frequently 
includes the largest percentage of computer code in a DSS 
and is most often modified [Sprague 82:p. 217]. Dialogue 
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management provides and manages communication between the 
decision-maker and a DSS. The form of communication often 
includes graphics in addition to text. 

There are several dialogue styles which may be found 
in a DSS. A common form is the question and answer 
dialogue. The DSS presents the user with a question, the 
user answers the question and the session continues until 
the DSS produces the answer to support decision-making. 

These dialogues are most helpful to inexperienced or 
infrequent users who are not familiar with the DSS. 

Another dialogue style is that of command language. 
Certain commands invoke DSS functions. For example, a CBT 
manager may enter a command such as PLOT LEARNING RATE and 
obtain a plot showing a student or class learning rate for 
all training course modules completed or in progress. 

A third common dialogue style is the menu. The user 
is presented with a number of alternatives from which to 
choose. Selection is made with a keyboard or another input 
device such as a mouse or light pen. There are also very 
limited applications of making a selection by touch-screen 
or voice instruction. 

It is very likely that a dialogue management 
component will mix or combine more than one dialogue style 
in any DSS. The objective is to keep the DSS compatible 
with the needs of the decision-maker. The user may actually 
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be able to choose a style of communication to suit his 
preference. [Sprague 82:pp. 199-205] 

3 . Model Management 

It is the model subsystem that gives decision-makers 
the opportunity to analyze a problem fully by developing and 
comparing alternative solutions. Decision models may be 
embedded in the DSS with the database capability as the 
integration and communication mechanism between them. The 
dialogue subsystem gives users direct control over the 
operation and manipulation of models. This link provides 
the ability for true interactive modeling. 

To serve DSS needs, the model subsystem must support 
decision-making activities such as projection, deduction, 
creation and comparison of alternatives, optimization and 
simulation. To provide these capabilities, a library of 
models comprising a model base must be established. The 
model base may be large or small and includes permanent 
models, ad hoc models, user-built or commercially obtained, 
”of f-the-shelf ” models. Models are included for 
operational, analytical and strategic decision support. 

Some models are used as ’’building blocks” to support the 
construction of other models. 

The comprehensive set of models required for 
decision support has led to the development of a Model Base 
Management System (MBMS). There is a pervasive analogy 
between the database with its DBMS and the model base with 
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its MBMS. The MBMS facilitates several general functions 
which are very important: 

1. Gener ate--cr eate new models quickly and easily; use 

model "building blocks” 

2. Update — change data in a model without restructuring 

the model 

3. Restructure — redefine model in response to changes 

in the modeled environment 

4. Catalog--categorize and maintain a wide range of 

models supporting all levels of management 

5. Integrate — access and link appropriate models 

6. Repor t--pro vide information for desired decision 

support 

Through the dialogue component, the decision-maker 
has the ability to interrupt the running of a model, run 
model segments and change sequence or parameters. Model 
management ensures that models draw inputs and parameter 
values from the database or databases and returns output to 
the applicable database(s). This promotes accuracy and 
consistency in model generated output for decision support. 
[Sprague 80:pp. 22-24, 82:pp. 257-272] 

C. PERFORMANCE OBJECTIVES 

Sprague [Sprague 80:pp. 18-20] delineates several DSS 
performance requirements. No particular DSS may be able to 
meet all requirements. DSS performance depends on the task, 
the organizational environment and the decision-makers 
involved . 
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A DSS should provide support primarily for semi- 
structured and unstructured decisions. For these decisions, 
databases and traditional information systems may be 
inadequate. DSS are designed to provide support for these 
kinds of problems. 

A DSS should provide support for managers at all levels. 
CBT managers from CNET to course instructors and training 
managers should find support from DSS to solve problems in 
their professional realm. 

A DSS should support decisions which are interdependent 
as well as those which are independent. Many decisions must 
be made in consonance with other decisions made by others or 
groups. The following decision types should be considered: 

1. Independent--a decision-maker has full 

responsibility and authority to make 
a decision 

2. Sequential — a decision-maker makes a decision or 

part of a decision which is then 
passed on to another decision-maker 

3. Pooled — a decision results from negotiation and 

interaction among decision-makers (A Group 
Decision Support System (GDSS) is a 
specialized DSS) 

A DSS should support all phases of the decision-making 
process. Herbert Simon [Sprague 80:p. 19] characterized 
three main steps in the decision-making process as 
intelligence, design and choice. Intelligence is monitoring 
or searching the environment for conditions calling for 
decisions. Raw data is gathered, processed and examined for 
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clues that may identify problems. Design is creating, 
developing and analyzing potential courses of action. 

Choice is selecting a course of action and implementing it. 

A DSS should support a variety of decision-making 
processes. Each CBT manager has an individual cognitive 
style and approach to problem-solving for his specific 
functional domain. There is no one best decision-making 
method. A DSS should be process independent and user 
controlled and directed. 

A DSS should be easy to use. It should not further 
complicate the decision-making procedure. The user must be 
comfortable with a DSS and develop confidence in it for it 
to become a valuable and convenient management aid. 

D. EXPERT SYSTEMS 

Expert Systems may lend considerable aid to the training 
process as well as its management. Although this technology 
appears to be in its infancy, there seems to be much 
potential to significantly improve and enrich the training 
process and the quality of training provided by CBT. 

An Expert System is a Knowledge Base Management System 
(KBMS) which is particularly useful for dealing with a 
diagnostic or other interpretative tasks [Mylopoulos 86:p. 
4]. Such a system attempts to mimic top human experts and 
embodies knowledge of a particular limited application area 
[Turban 85:p. 138]. The role of the Expert System in 
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decision-making is extracted from Figure 4.1 and illustrated 
in Figure 4.2. 

Expert Systems have evolved from earlier Management 
Information Systems (MIS) and often interact with them. 

For example, DBMS provide information to Expert Systems in 
the same fashion as a human expert would use a data source 
such as a book or library to obtain factual information. 
Expert Systems may also rely on the model base of a DSS to 
provide established mathematical and forecasting models to 
lend support for system decision-making and expertise. 
[Turban 85:pp. 145-146] However, there are significant 
differences between a DSS and an Expert System as 
illustrated in Table 4.1. 

According to Bevan and Wetherall [Bevan 87:p. 21], the 
process of building the knowledge base associated with an 
Expert System is long and complex but contains very valuable 
information. Most Expert Systems are designed to be used in 
a consultative manner by non-experts to perform a task in 
the narrow area of expertise. In the consultative role. 
Expert Systems may aid a technician in equipment fault 
isolation and suggest appropriate corrective action. 

However, Bevan and Wetherall suggest that the idea of using 
the same knowledge base in a training role is very 
attractive . 

The Air Force Human Resources Laboratory (AFHRL) has a 
research program to develop and demonstrate the technology 
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Information System (Expert System) Decision- 

maker 

Figure 4.2 Information System and Decision-maker 
Relationship for Expert System 
[Mason 69:p. 86] 

for an Intelligent Maintenance Advisor. The Maintenance 
Advisor would serve as an "expert’* for both consultative and 
training roles. As envisioned, the Advisor would be a skill 
integrator by using and capturing the corporate memory of 
skilled technicians. This knowledge would be used in turn 
to provide trouble-shooting training for novice technicians. 
[Richardson 86:pp. 47-50] 

1 • Varieties of Knowledge 

Knowledge may be classified in several ways. As 
illustrated in Chapter III, knowledge may be comprised of 
the complimentary ingredients of formal training and prior 
experience. As a result of formal study, knowledge is 
gained which is composed of definitions, axioms and laws. 
Successful trainees emerge from courses with a firm grasp of 
the terms, procedures and laws which constitute the formal 
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TABLE 4.1 DIFFERENCES BETWEEN DSS AND EXPERT SYSTEMS 

[Turban 85:p. 141] 





DSS 


Expert System 


Objective: 


Assist human 


Replicate (mimic) 
human 


Who makes the decision? 


Human chooses 
from options 


Human considers a 
recommendation 


Major orientation: 


Decision making 


Transfer of 
expertise 


Problem area: 


Complex, wide, 
integrated 


Narrow domain 


Query direction: 


Human queries 
system 


System leads 
human 


Database : 


Factual 

knowledge 


Procedural and 
factual 
knowledge 



theories and accepted principles of their training 
discipline. This type of "head" knowledge can readily be 
described but has yet to be applied in a practical way. 

[ Harmon 85 : p . 30 ] 

This kind of knowledge is generally well-structured 
and is amenable to the formulation of general principles for 
correct behavior. Spiro states that it is this knowledge 
domain in which past educational practice has had the most 
success and cognitive psychology and artificial intelligence 
have made their greatest advances. [Spiro 87:p. 286] 
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Knowledge may also be compiled by personal 
experience or by learning from a mentor. The student learns 
to rely on r ules-of-thumb to perform specified tasks or to 
solve simple problems. This type of knowledge is referred 
to as heuristics. [Harmon 85:p. 31] Knowledge of this type 
may be relatively ill-structured. It may be difficult to 
define a single set of attributes for all cases. 

Interactions between various features in complex problems 
proliferate and may be highly dependent on one another. 
General principles are insufficient to solve individual 
cases satisfactorily, resulting in problems which are very 
difficult to decipher and resolve. [Spiro 87:p. 286] 

2 . Expertise 

How do experts solve problems? They use expertise 
gained about a subject to solve problems in that realm. An 
expert is an individual who is recognized as being able to 
solve a particular type of problem that most other people 
cannot solve as efficiently or effectively. Experts are 
able to perform successfully because they have acquired a 
large amount of domain-specific knowledge--exper tise . 

Expertise may be gained entirely from experience or 
wholly from education without the benefit of formal 
training. However, most experts, including pilots, doctors, 
accountants and technicians, begin by studying their 
specialty in school. They learn principles and theories 
which are considered as basic to their profession. They 
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continue by gaining experience through working in their 
professions and perhaps, working with a mentor. In this 
way, they gain expertise and may become regarded as expert 
in their profession. [Harmon 85:pp. 31-32] 

E. EXPERT SYSTEMS AND COMPUTER-BASED TRAINING 

Experience-Consolidation system is a term which 
describes an Expert System intended primarily for training. 
These systems ’’telescope’* aspects of the process by which 
practical field experience leads to the acquisition of 
expertise. Thus an expert methodology is scrutinized to 
determine and then teach its essential characteristics. 

These systems attempt to teach intuition and give people the 
ability to think correctly for themselves without explicit 
guidelines in particular realms. Experience-Consolidation 
systems attempt to shorten the time required for the would- 
be technician to gain experience from years to months or 
days. These systems attempt to teach what has not been 
taught conventionally and has traditionally been left for 
”on- job-training” (OJT) or as a skill to be gained 
through ’’field experience.” [Spiro 87:p. 285] 

Experience-Consolidation systems most closely resemble 
the Computer-Aided Instruction (CAI) form of simulation. 
Simulation does require active participation by students and 
promotes learning by doing. However, there are significant 
differences between simulation and Experience-Consolidation 
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systems. Simulation usually does not convey all the 
multivariate attributes of a complex case. It is not 
concerned with the effective acquisition of multi-patterned 
information. And, simulation cannot rapidly present a large 
number of cases for consideration. Experience-Consolidation 
systems and simulation may be used to complement each other 
in a training environment. [Spiro 87:p. 291] 

1 . Implementation 

As noted above, the idea of using Expert Systems in 
a tutoring role is very attractive. Is this possible? Is 
it practical? Expert Systems for training or Experience- 
Consolidation systems attempt to shorten considerably the 
time required to acquire large amounts of experience and 
expertise and then pass these on to trainees. The process 
must safeguard the essential characteristics of the 
expertise so that it remains relevant. 

A dilemma is obvious. In the real world, these 
essential characteristics themselves hinder shortening of 
expertise acquisition time. First, actual cases occur over 
long duration. Also, the amount of information to be 
gleaned over time is large and by definition, ill- 
structured. The information cannot be arranged (chunked) in 
hierarchical fashion and there is simply too much to 
remember. Finally, cases happen in haphazard order without 
regard to instr uc t ionally useful arrangement. [Spiro 87:p. 
287] 
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Spiro [Spiro 87:p. 287 ff] discusses each of these 
problems and suggests ways in which they may be overcome. 

To use this technology successfully, a training program must 
solve the challenges presented by these difficulties. The 
following sections present several of these ideas. 

a. The Problem of Time 

The experience needed to gain acceptable 
expertise may last for years. This problem is addressed by 
gathering many cases and extracting from them the most 
significant multivariate information needed to fully 
characterize a case’s environment. For the Navy Maintenance 
Training Improvement Program (MTIP) , this could be 
accomplished by gathering many skilled technicians, regarded 
as experts in the applicable field, and conducting 
interviews or making observations. Why are exhaustive 
representations important? The need is to richly and fully 
present the context in which the decision factors occur. A 
principle of Experience-Consolidation systems is that in 
ill-structured domains, contexts are taught to teach 
concepts. Situations (cases) are thus presented from many 
points of view. 

b. The Problem of Information Overload 
This problem is the most difficult. The 

challenge is met by using specially designed graphic aids 
which are tailored to simultaneously present complete sets 
of complexly entangled patterns at a glance. This method 
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taps into everyone’s finely developed perceptual processing 
system. Experimental psychologists have established that 
humans have an extraordinary ability to remember many 
briefly presented visual scenes. The message for CBT is 
clear. High quality graphics capability is essential for 
this type of training. This powerful ability of human 
thought is employed to encode the manifold features of ill- 
structured conceptual situations. These situations can only 
be understood and comprehended by simultaneous appreciation 
of multiple, partially overlapping interrelationships. 

This perceptual solution to the information 
overload and memory problem provides an optimal 
representation for conveying pattern and context 
information. A perceptual view comprehends many things 
juxtaposed. In a short time, many coexistent parts are 
viewed in one field of vision. Consider all the information 
absorbed simultaneously by viewing the image of a face or a 
landscape. People readily perform on stored perceptual 
information to detect patterns of resemblance and 
dissimilarity. Experience-Consolidation systems use the 
computer to tap into a powerful processing system of humans. 
This system is well suited to processing large amounts of 
intricately entangled information in ill-structured 
knowledge domains. 
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c. The Problem of Arrangement 

An arrangement system must be designed to 
maximize the retention of robust case information in order 
to highlight case information which may produce application 
difficulties. The system must be able to distinguish cases 
which appear similar but produce different outcomes and 
those which seem unrelated but produce similar outcomes. An 
arrangement system must also classify cases so that many 
complex resemblance patterns within the same outcome 
category can be learned. The power, speed and flexibility 
of a computer are needed to make a functional system. 
Highlighting multiple connections and similarities between 
cases, permits de-emphasis of the retrieval of precompiled 
diagnostics in favor of diagnostic building, to fit the 
needs of a given situation. If training is to transfer 
expertise, the flexibility of knowledge representation is 
crucial . 

2 . An Intelligent Computer-Aided Instruction 
System Model 

An Expert System which is employed for training may 
be referred to as an Intelligent Computer-Aided Instruction 
(ICAI) system. According to Bevan and Wetherall [Bevan 
87:p. 21], with this technique, knowledge bases are used to 

represent the teaching domain and to govern the operation of 
the instructional system. There are two advantages to this 
approach : 
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1. The operational characteristics of the system may 
be easily modified by using a rule base to govern 
the operation of the instruction system. 

2. Isolation of domain dependent rules from 
instructional rules permits the creation of a 
generalized ICAI system which can be used for a 
variety of teaching domains. 

This is seen as a way to capture the knowledge base 
of an Expert System for training and provides a means 
whereby operational staff can observe the performance of an 
Expert System. Such a system has been created and is known 
as TUTOR. 

A model of the TUTOR System is illustrated in Figure 
4.3. In simplified terms, the system makes a current 
assessment of a student's ability and knowledge from the 
Student Model. The Teaching Generator then decides what 
kind of interaction (exposition or problem to answer, for 
example) to give the trainee and selects a particular topic 
to present. The Administrator presents the situation to the 
student through the Natural Language Interface and to the 
Domain Expert System for the correct solution. Finally, 
each interaction with the trainee is recorded in the Student 
Model, so that the Teaching Generator may choose what to do 
next. Note that the TUTOR is an Expert System in its own 
right. Its major components are more fully described in the 
following paragraphs. [Bevan 87:p. 22-23] 
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Student 



Figure 4.3 Intelligent Computer-Aided Instruction 
System Model (TUTOR) [Bevan 87:p. 23] 



a. Administrator 

The Administrator serves as the nerve center of 
the system. It provides general management and control over 
system resources. It serves as the primary connection and 
main interface between components of the system. 
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b. Domain Expert System 

This component contains the domain specific 
rules as they would appear in a consultative Expert System. 
It provides information to check student answers and 
responses. An explanation facility should be included to 
provide trainees with the reasons for correct answers or 
responses . 

c. Natural Language Interface 

Compared to a consultative system, this 
component is more important for the training system. A 
teaching system will require more two way communication 
between student and machine. Sometimes the student will be 
controlling the dialogue such as when seeking further 
explanation about subject material, at other times the 
system will be presenting material and controlling the 
dialogue. A fairly sophisticated Natural Language Interface 
is needed to give a wide range of users freedom in 
expressing their responses or answers to problems. 

Another requirement of the Natural Language 
Interface in a generalized system is that it be able to 
modify the vocabulary when the subject domain is changed. 
This is important for technical training subjects and to 
accommodate unique military terms. 

The U. S. Navy has sponsored research to develop 
an "intelligent” interface to CBT. The interface would be 
capable of responding to questions and answers in free form. 
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Humans could communicate with computers in English without 
having to learn a special computer language. Furthermore, 
the knowledge base used for response generation could be 
automatically expanded as trainees and course administrators 
interact with the system. [Bryant 88:pp. 199-206] 

d. Student Model 

The Student Model maintains a record of a 
student’s knowledge of the subject domain. It may contain 
information about a student’s preferred learning strategy 
and level of expertise. Levels of expertise may be 
categorized into several classes such as novice, competent 
but needing revision, skilled or expert, for example. 

e. Teaching Generator 

The role of this component is similar to that of 
a human tutor. It provides initial, corrective and remedial 
training on individual topics in the syllabus. To determine 
trainee comprehension, it generates problems for the student 
to solve consistent with the student model and the 
curriculum material as presented. It determines the nature 
of problems for consideration and the method of 
presentation. [Bevan 87:pp. 22-25] 

F. SUMMARY 

The chapter began with a discussion of DSS and Expert 
System applications to training. The major subsystems of a 
DSS--data management, dialogue management and model 
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management — were described. Several performance objectives 
were discussed. A DSS must be easy to use, promote user 
confidence and not further complicate decision-making. An 
effective DSS will accommodate the needs of all levels of 
training management. Also, a DSS should provide assistance 
with both independent and interdependent decisions which 
lead to effective and efficient use of training resources. 

This chapter portrayed the Expert System as an 
powerful vehicle for training. Expert Systems have 
traditionally been relied on in a consultative manner to 
provide solutions for specific problems in the realm of 
expertise. The knowledge encapsulated by these systems 
makes them attractive for technical training. However, 
several major difficulties are apparent. The problems of 
time, information overload and arrangement are presented. 

Nevertheless, Expert Systems have been applied to 
training programs. A model of one such implementation, 
TUTOR, is illustrated. The system itself functions as an 
Expert System. It has five major component s--Administrator , 
Domain Expert System, Natural Language Interface, Student 
Model and Teaching Generator. The roles and functions of 
each component are discussed. Although in its infancy, this 
technology may provide significant GET potential with 
specific application to the Maintenance Training Improvement 
Program (MTIP) as discussed in the following chapter. 



80 



V. COMPUTER-BASED TRAINING APPLIED TO THE 
MAINTENANCE TRAINING IMPROVEMENT PROGRAM 



A. INTRODUCTION 

As described in previous chapters, the U. S. Navy has 
had a long association with various aspects of Computer- 
Based Training (CBT), In recent years, the Maintenance 
Training Improvement Program (MTIP) has become very 
important in the training of naval aviation maintenance 
technicians. For the MTIP, most data for learning and 
student information is computerized. 

MTIP is a training management system and a method of 
inservice training. The program is a command responsibility 
which attempts to identify maintenance performance 
deficiencies at the command, work center and individual 
technician levels. As deficiencies are identified, 
refresher or remedial training is conducted to provide 
correct technical knowledge and to improve job skills. [NAMP 
86:pp. 12-7 and 12-8; CNAP 86:p. 1] 

The primary means used to identify knowledge and skill 
deficiencies is through a testing program which includes 
standardized question and answer banks from which 
computerized tests are generated. The tests are intended to 
examine the knowledge and skill of maintenance technicians 
compared to their peers and standards established for each 
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occupational specialty or rating. Test questions are to be 
prepared by Subject Matter Experts (SME) and require 
periodic validation. [CNAP 86:p. 1] 

MTIP has been widely implemented and provides units with 
some flexibility to direct their own inservice training 
program and compare the outcomes to established standards. 
Although use is widespread, acceptance has been less than 
desired. The program is seen as administratively burdensome 
and not producing intended results, such as, significantly 
improved maintenance technician knowledge and skill levels. 
It is the thrust of this thesis that application of CBT 
technology, as discussed in previous chapters, may provide a 
means to improve the current MTIP and lead to future 
enhancements . 

B. COMPUTER-BASED TRAINING TECHNOLOGY FOR MTIP 

CBT technology involves much more than the physical 
components that are visible and obvious. It encompasses 
methods, techniques and procedures that result from 
tradition, regulations, vision and available resources. 
Technology may be divided into ’’hard” components (physical 
devices) and ’’soft” components (methodologies). While those 
components may be identified individually, they are closely 
intertwined in actual technology applications. When 
evaluating a training system, the interaction among 
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components must be considered as well as the components 
themselves. [Kearsley 84:p. 3] 

The MTIP is conducted mostly by aviation squadrons, 

Fleet Readiness/Replacement Aviation Maintenance Personnel 
(FRAMP) units, Naval Air Maintenance Training Group 
(NAMTRAGRU) units. Navy Engineering Technical Services 
(NETS) or Contractor Engineering Technical Services (GETS). 
This training is primarily passive and relies largely on 
static media such as printed technical manuals, workbooks 
and slides. There is also some use of dynamic media. This 
includes films and video. The use of an interactive media, 
i.e., a computer, for actual training does not exist in the 
MTIP. Computers have been used to maintain databases and 
administrative records. CBT does provide the capability for 
active learning with computers. 

1 . Database Management 

Chapter III describes why data is a very important 
resource in a CBT system. It is important because of the 
information which becomes available to both students and 
managers. This is certainly true for MTIP and is an area 
where significant progress could readily be made. 

The MTIP has been perceived as producing too much 
inaccurate, misleading or wrong information. Much of this 
problem relates to the system development and is discussed 
later under Instructional Systems Development (ISD). 

However, an integrated computer-based system could be more 
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responsive to the users* It is now very difficult and time 
consuming for users to provide corrective input to the 
existing system. 

A Database Manager, similar to a Database 
Administrator (to use database jargon), should be 
established for each MTIP database. The Database Manager 
should have the power to control that database and not 
merely coordinate or monitor activity. Users should be able 
to provide change inputs quickly and preferably on-line. 

The Database Manager should be able to promptly evaluate the 
inputs and make changes as necessary. Proper evaluation of 
change recommendations will require the many resources 
including those of Naval Air Systems Command (NAVAIR), GETS, 
NETS and occasionally the commercial contractors of the 
system, aircraft or equipment involved. The Database 
Manager should be able to determine a proper solution for 
change recommendations and provide rapid response to a user 
and alter the database as necessary. 

The database must be very large to encompass the 
scope and depth of all data needed for complete maintenance 
training for even one type of aircraft. If the database is 
too limited students quickly learn the ”right” answers to 
particular questions. As this **gouge” is passed to others, 
the MTIP looses effectiveness. A more advanced system 
should be able to produce differing questions and answers on 
the same topic. 
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The MTIP database could store appropriate lessons or 
lesson guides for MTIP topics in addition to question and 
answer banks. It has been difficult to coordinate training 
with established testing. CBT could improve the 
interconnection between training and testing materials. The 
training materials could be generated as they currently are 
by FRAMP, NAMTRAGRU and squadron personnel and be 
incorporated into the MTIP databases. 

With the right equipment, training could be 
conducted at the squadron without having to schedule 
training sessions with another activity. Training could 
also be conducted on a more individual basis so that entire 
work centers would not have to be absent for training at the 
same time. Individualized training has other benefits which 
are discussed later. Additionally, CBT could better 
facilitate remedial training sessions at a FRAMP or 
NAMTRAGRU. This would provide better coordination and 
improved use of training resources over a manual system, 

2 . Decision Support 

Chapter IV describes the fundamentals of a general 
Decision Support System (DSS) and provides some specific 
applications to the CBT environment. DSS are created to 
provide assistance for semi-structured and unstructured 
decisions. For the MTIP to be a true Maintenance Training 
Improvement Program, it must assist the training manager in 
decision-making to further improve and refine the MTIP, 
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For example, based on MTIP data, managers should be 
able to develop student profiles and be able to classify 
students into desired categories such as novice, apprentice 
or master technician. A DSS could provide an optimal 
training plan for each category of trainee. With 
information of this type, remedial training could be 
developed to better suit the needs of individual students. 
This could provide more stability and direction to a 
training program. 

A DSS function can also lead to better resource use. 
Training resources of several activities are now available 
to the MTIP. A centralized, responsive system could aid the 
decision-maker in the most efficient coordination and 
scheduling of those resources with user need and priority 
considered appropriately. Additionally, usage data of the 
various resources would be available to support training 
managers in decisions about redistribution of existing 
assets or further development and procurement of new 
hardware, software or training technology. 

A DSS can also serve to give indications of overall 
MTIP effectiveness. As criteria are incorporated, 
information on various aspects of effectiveness can be used 
for evaluation. "What if?" evaluation may be done for 
squadron, functional wing or type command changes. 
Appropriate corrective action may be initiated to improve 
effectiveness . 
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A DSS as part of the MTIP, must use data which is 
easily available and obtainable. It must be designed to 
produce output which is meaningful, timely information, not 
just data. The DSS must focus on the where, what and when 
of technical training for the various specialties or 
ratings. [Ruck 86:pp. 47,54] 

3 . Expert System 

Expert Systems have typically been considered useful 
for consulting or as advisors. In these roles, the Expert 
System is relied upon to provide expertise for a specified 
topic. As described in Chapter IV, the idea of using an 
Expert System knowledge base for training is very 
attractive. Attempts have been made to convert or develop 
effective training tools from consultative Expert Systems. 
[Clancey 87:pp. 1 ff, Richardson 86:pp. 47-51] 

There are several possible uses of Expert Systems 
for future MTIP development. Expert System may be able to 
impart expert knowledge to maintenance technicians. This is 
much more than the information taught by the current MTIP. 
Expertise can now be gained only through years of job 
experience, on- job-training (OJT) and study. There is 
potential for an Expert System to capture an aggregate of 
existing expertise for various ratings. Using Expert System 
training, expertise may be transferred to students in days 
and months instead of years. 
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Furthermore, an Expert System may raise 
individualized training, a key concept of the MTIP, to a new 
level. It can provide unique initial, corrective and 
remedial training for each student based on the student 
profile. It can "intelligently” determine trainee 
comprehension, generate problems for the student and 
determine the method of presentation. Each trainee receives 
instruction designed specifically for him. Another feature 
of an "intelligent" system is that it can respond to a much 
wider range of communication with humans. It can make a 
system much more "user friendly." The U. S. Navy has 
sponsored research to develop an "intelligent" interface to 
CBT [Bryant 88:pp. 199-206]. Future developments may have 
application for the MTIP. 

CBT offers the possibility of considerably expanding 
the scope of the current MTIP. The Air Force Human 
Resources Laboratory (AFHRL) has made an initial effort to 
develop an Intelligent Maintenance Advisor. The envisioned 
role of the Advisor is to (1) enable unskilled technicians 
to perform as if they were skilled ("skill multiplier"), (2) 
tutor technicians who need further development of their 
technical skills and (3) work cooperatively with skilled 
technicians to capture diagnostic insights (expertise) for 
the benefit of their successors. Such a device retains its 
training function and also serves as a "skill integrator" to 
interactively assist the experienced technician with 
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possible solutions to a problem being considered. Finally, 
this technology could provide the MTIP with an authoring 
system to create new training knowledge base or modify 
existing knowledge bases. [Richardson 86:pp. 47-48] 

4 . Instructional Systems Development 

A common methodology for conducting CBT is 
Instructional Systems Development (ISD). The Commander, 
Naval Air Force, United States Pacific Fleet, has directed 
that test questions be generated for MTIP using ISD [CNAP 
86:p. 1]. The most important goal of ISD is to provide 
performance oriented training. 

CBT is the most feasible method to support an 
extensive ISD framework for a large scale training program 
like MTIP. The performance oriented philosophy differs from 
the traditional curriculum approach. ISD focus on providing 
consistent training to improve future job performance while 
the traditional approach centers on the training of topics 
developed individually each time the training is offered, 
without regard to how the training will be used. Table 5.1 
illustrates the difference between MTIP using the ISD and 
traditional approaches to instruction. [Kearsley 84:pp. 83- 
84] 

Another characteristic of ISD applicable to MTIP, is 
that empirical evaluation is an integral part of the 
training program. In the course of training, the user and 
the managers may detect deficiencies in the program. Also, 
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the needs of the user or the Navy may change with time. A 
computer-based training program may be altered and 
customized to meet the needs of the Navy or users. 
Corrections may be implemented promptly. Service-wide 
alterations may be incorporated rapidly with good 
configuration control and high confidence of standardization 
where needed. 

A third characteristic of ISD is the attempt to 
define optimal strategies, media and sequencing for selected 
training activities. While MTIP does seek to find the 
optimal strategy to conduct training, the use of limited 
media and inability to optimally control sequencing also 
apply. CBT can coordinate the availability of many media-- 
print, slides, video, computers, simulators, embedded 
training and expert systems--to illustrate a range of 
possibilities. [Kearsley 84:p. 83] 

ISD may be applied to both computerized and non- 
computerized training. MTIP has employed the most basic 
features of CBT. Computerized impact has primarily been 
along the lines of Computer Managed Instruction (CMI) to 
meet administrative needs. Future developments and 
enhancements for MTIP will likely involve more application 
of CBT and are discussed later. The large size of MTIP, 
potential high cost for further evaluation and maturation, 
and the commitment to ISD suggest a phased approach with 
increased CBT for MTIP. 
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TABLE 5.1 COMPARISON OF MTIP WITH ISD AND TRADITIONAL 
APPROACHES TO INSTRUCTION [Kearsley 84:p. 84] 



MTIP using ISD 



Traditional 



Content based upon 
performance 

Instructional objectives are 
explicitly stated 

Content is developed by 
multidisciplinary team 

Evaluation is part of 
development process 

Media, strategies and 
sequencing are analytically 
derived 

Materials take into account 
student capabilities and 
dif f er ences 

Development process and 
implementation are consistent 
and reliable across 
individuals 



Content based upon topics to 
covered 

Instructional objectives are 
implicit 

Content is developed 
primarily by instructor 

Evaluation occurs after 
training is implemented 

Media, strategies and 
sequencing are intuitively 
selected 

Materials are designed for 
ideal or average student 

Development process and 
implementation are 
idiosyncratic and vary with 
instructor 



ISD has been most widely and thoroughly applied in 
the military training environment. This is because of the 
long lead time and expense required for the most effective 
training program and the severe consequences of inadequate 
training programs. Table 5.2 depicts the benefits to MTIP 
of the ISD methodology compared to traditionally developed 
instruction. [Kearsley 84:pp. 87-88] 
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TABLE 5.2 BENEFITS FOR MTIP FROM ISD COMPARED TO 
TRADITIONALLY DEVELOPED INSTRUCTION 
[Kearsley 84:p.88] 



1. Training closely matches job requirements and 

student needs 

2. Training takes significantly less time to complete 

3. Training is highly consistent across different 

occasions and instructors 

4. Training can meet a wide range of variation in 

student entry levels and abilities 

5. Improved confidence, morale and satisfaction on the 

part of students 



ISD has been described as consisting of five major 
phases which are illustrated in Figure 5.1. Each phase may 
include several steps depending upon the application. 
[Kearsley 84:pp. 85-86, Logan 82:pp. 6-12] These phases are 
relevant to MTIP development and enhancement, 
a. Analysis 

In the Analysis phase, needs are assessed with 
respect to a performance problem or a new requirement. The 
nature and scope of the training required are identified. 

Is MTIP training the appropriate solution? Perhaps a 
personnel, organizational, procedural or job aid solution is 
needed. Also, available resources and the time and schedule 
constraints for a prospective MTIP must be identified. 
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ANALYSIS DESIGN DEVELOPMENT IMPLEMENTATION 




H 

< 

< 

> 

w 



93 



Figure 5.1 Major Phases o£ ISD Applicable to MTIP 
[Kearsley 84:p. 85, Logan 82:p. 8] 



Additionally, the jobs or tasks to be taught roust be 
identified by the functions to be perforroed. Finally, 
students to be trained should be analyzed to determine 
aptitudes, motivation, reading ability and prior job 
knowledge and experience. Can individual differences be 
accommodated by MTIP training? 

b. Design 

The Design phase is concerned with defining 
instructional objectives, specifying lessons and tests and 
selecting the applicable media. Instructional strategies 
and sequencing based on aircraft systems are determined. In 
this phase the nature of the training can be specified. 

c. Development 

Lessons and tests are written based on input 
from aircraft maintenance experts. This involves technical 
writing and the production of manuals, slides, video or CBT. 
Also, prepared are instructor guides and materials for 
administration of MTIP such as training plans, evaluation 
forms and student records. 

d. Implementation 

Implementation is the delivery and management of 
training. Instructors and staff must be trained to use CBT 
such as the MTIP. Facilities are readied while time and 
space are allocated for training. Training is conducted, 
students learn, take tests and receive remedial training as 
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necessary. Records are kept and data is collected, 
distributed and stored. 

e. Evaluation 

As shown in Figure 5.1, Evaluation occurs 
throughout all phases of ISD. Evaluation provides feedback, 
control and regulation for the entire process. CBT can 
provide this capability. Initially, evaluation is made to 
improve instruction (formative). Later, judgments are made 
about the effectiveness of training (summation). As 
Evaluation controls the ISD process, it may also trigger the 
beginning of a new process or the termination of ISD for a 
particular environment. [Kearsley 84:pp. 85-87, Logan 82:pp. 
6-13] 

C. SUMMARY 

This chapter portrayed application of Computer-Based 
Training (CBT) to both the present and future Maintenance 
Training Improvement Program (MTIP). The present MTIP was 
described briefly. Several proposals were made to improve 
the current MTIP through better database management 
techniques. Progressive database management provides a much 
more responsive system and one that is more in tune with the 
needs of user operators. 

Decision support available in a CBT system, could 
provide MTIP managers and users with much more suitable 
information with which to manage. With this support, MTIP 
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resources could be better used to meet training needs. With 
efficient resource use and better management information 
readily available, MTIP effectiveness will certainly improve. 

Expert Systems were described as a means of significantly 
altering the training process itself. It may become 
possible to teach experience as well as information. Expert 
Systems may permit more integration of actual maintenance 
and the training to do maintenance. CBT has great potential 
to provide enhancements to the current MTIP and promote 
progressive future developments. 

Finally, Instructional Systems Development (ISD) was 
presented as the preferred method of MTIP with CBT 
implementation. In fact, current policy directs that test 
questions for current MTIP applications be generated with 
ISD. Some of the characteristics of ISD are described and 
the five major phases are illustrated. ISD was depicted as 
being performance oriented and several of the major benefits 
for MTIP are portrayed. 
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VI. CONCLUSIONS AND RECOMMENDATIONS 



The installation of computers into the classroom alone 
does not result in Computer-Based Training (CBT). Rather, 
CBT draws from both educational and computer technology to 
add new dimension to training. CBT has been implemented for 
a number of reasons. These reasons may generally be 
categorized as those which reduce training costs and/or 
those which improve training effectiveness. CBT has not 
been a panacea for all training environments but has worked 
well in many military training centers. 

The Maintenance Training Improvement Program (MTIP) is 
already a computer-based training program which can benefit 
substantially from increased employment of CBT! MTIP can be 
impacted by CBT in the two areas of database processing and 
management and, secondly, decision systems. 

A. DATABASE PROCESSING AND MANAGEMENT 

Database processing is an important characteristic for 
CBT. A typical military training program has a formidable 
record-keeping requirement which may not be adequately 
recognized. Modern database processing streamlines record- 
keeping and facilitates training and training management by 
providing the most relevant data and information as needed. 
Furthermore, a sound Database Management System (DBMS) 



97 



establishes a foundation for additional computer-based 
enhancements for CBT. 

The MTIP needs Database Managers who have a sense of 
ownership for their databases and maintain empathy with the 
users. This attitude will result in a training program 
which is robust and responsive to operator inputs and needs. 
Databases must be rapidly up-dated as change recommendations 
are approved or new data and procedures become available. 
MTIP databases should serve as libraries for their 
respective type of aircraft. 

B. DECISION SYSTEMS 

CBT can incorporate features of a Decision Support 
System (DSS). Managers of CBT are required to make 
decisions regarding CBT and students. Many decisions are 
routine and well-structured. A DSS; however, lends aid for 
those training management decisions which are unstructured. 
All levels of management and many types of decisions are 
supported by DSS leading to a more conducive training 
environment for students enrolled in CBT. 

MTIP managers need routine structured reports to control 
the daily operations of the program. A DSS capability would 
also provide support for resource optimization concerns and 
future oriented considerations. DSS provide mathematical, 
operations research and statistical tools to aid managers 
with these types of unstructured decisions. With this kind 
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of support, managers can maintain a state-of-the-art system 
by optimizing available resources and choosing the needed 
hardware, software and training technology which will bring 
maximum future benefit. 

Expert Systems have found limited application to date in 
CBT. Their potential, however, is very significant. Expert 
Systems offer the possibility of teaching another dimension 
of knowledge, namely, heuristics. These systems can help 
students learn in a classroom from the experience and 
expertise of others. This can provide students with a level 
of knowledge which has traditionally not been taught by 
conventional methods. 

The MTIP is an excellent candidate for Expert System- 
based training. It does not concentrate on topical 
classroom training; rather, it is action and performance 
oriented. The MTIP does teach basic facts and procedures, 
but its major focus is to train students to think and act 
like successful, experienced aircraft maintenance 
technicians . 

C. MAINTENANCE TRAINING IMPROVEMENT PROGRAM MANAGEMENT 

It is apparent that equipment and systems to be 
maintained by military personnel are becoming more 
sophisticated and technically advanced. CBT offers the 
potential to bring substantial computing power to personnel 
training. For CBT to be effective it must be regarded as a 
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new technology. It is not merely a combination of existing 
educational and computer technologies. The adequacy and 
quality of CBT depend on the effort which goes into its 
planning, design, development and implementation. The 
various aspects of computing power--da tabase processing and 
management, decision support and Expert Systems — bring a 
synergy to CBT which can dramatically improve the training 
provided. 

The primary thrust is that CBT may be more fully applied 
to the Maintenance Training Improvement Program (MTIP). 

Both the current MTIP arrangement and future developments 
and enhancements can benefit greatly from a systematic CBT 
approach. Considerable work remains to develop CBT 
technology for MTIP to the level described here. Decision 
systems for training are in their infancy. Personnel 
policies must ensure the adequate recruitment and assignment 
of people to do MTIP development, implementation and 
maintenance. Sites must be selected to test prototype 
developments . 

Thus, naval aviation maintenance technicians can be 
better prepared to meet the challenges of an increasingly 
complex environment where high, sustained equipment and 
system readiness is expected and required. 
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APPENDIX A 



DEFINITION OF TERMS 



There is a proliferation of terminology associated with 
CBT. The field is made up of people and commercial vendors 
with very diverse backgrounds. CBT also has a wide variety 
of applications and considerable variability in the use of 
terms is found in the literature. Some of the most common 
terms are listed to explain their meaning and the concepts 
which the terms represent [Burke 82;pp. 180-181]. 
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DEFINITION OF TERMS 



Automated Instruction (AI) Any type of instruction in 

which administration or delivery is controlled by a 
computer . 

Automated Teaching (AT) See Automated Instruction. 

Automated Training (AT) See Automated Instruction. 

Computer-Aided (Assisted) Instruction (CAI) An 

application of Computer-Based Training that involves an 
on-line interactive learning process where a computer 
is the primary delivery system. 

Computer-Aided (Assisted) Education (CAE) See Computer- 

Aided Instruction. 

Computer-Aided (Assisted) Learning (CAL) See Computer- 

Aided Instruction. 

Computer-Aided (Assisted) Training (CAT) See Computer- 

Aided Instruction. 

Computer-Based Education (CBE) See Computer-Based 

Training . 

Computer-Based Instruction (CBI) See Computer-Based 

Training . 

Computer-Based Training (CBT) An interactive computer 

training system in which a computer is the primary 
delivery system and is used to provide overall training 
management . 

Computer-Managed Education (CME) See Computer-Managed 

Instruction . 

Computer-Managed Instruction (CMI) An application of 

Computer-Based Training which uses a computer to direct 
students through their training and produces reports on 
student performance or system utilization. It has 
elements in common with Computer-Aided Instruction, but 
Computer-Managed Instruction usually refers to a stand- 
alone system. 

Computer-Managed Learning (CML) See Computer-Managed 

Instruction . 
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Computer-Managed Training (CMT) See Computer-Managed 

Instruction, 

Conventional Instruction (Cl) Classroom, laboratory or 

lecture instruction which is presented to a group of 
students as a whole. Students usually have similar 
academic aptitudes and all receive the same 
instruction. Instruction is paced at a rate to 
accommodate all or nearly all students. 

Embedded Computer-based Training Interactive training 

which is an integral part of equipment or a system. 

Individualized Instruction (II) Instruction which 

accommodates individual differences in aptitude, skill 
level, prior knowledge and learning style. It is 
characterized by minimal time constraints, choice of 
instructional medium and adjustments to the training 
program as it progresses based on feedback. 

Intelligent Computer-Aided Instruction (ICAI) A 

Computer-Aided Instruction (see) system implemented by 
Expert System knowledge base techniques. 

Programmed Instruction (PI) Instruction in which 

material is presented cumulatively in small steps and 
progresses as the student masters each step. It may be 
presented by either computerized or noncomputerized 
media . 

Self-Paced Instruction (SPI) Instruction which allows 

students of different aptitudes and prior knowledge to 
progress through a training program at their own rate. 
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APPENDIX B 



ACRONYMS 

There are many acronyms associated with Computer-Based 
Training (CBT) and computer technology. A military 
application may introduce unfamiliar acronyms for activities 
as well as systems or concepts. A reference source is 
provided to specify and clarify acronyms applicable to 
military CBT. 
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ACRONYMS 



AFHRL 


Air Force Human Resources Laboratory, 
Brooks Air Force Base, Texas 


AIS 


Advanced Instructional System 


CAI 


Computer-Aided Instruction 


CBT 


Computer-Based Training 


CCP 


Communication Control Program 


GETS 


Contractor Engineering Technical 
Services 


CNET 


Chief of Naval Education and Training, 
Pensacola, Florida 


CMI 


Computer-Managed Instruction 


DBMS 


Database Management System 


DSS 


Decision Support System 


EDP 


Electronic Data Processing 


FRAMP 


Fleet Readiness /Replacement Aviation 
Maintenance Personnel 


ICAI 


Intelligent Computer-Aided Instruction 


IDA 


Institute for Defense Analysis, 
Arlington, Virginia 


ISD 


Instructional Systems Development 


KBMS 


Knowledge Base Management System 


MBMS 


Model Base Management System 


MIS 


Management Information System 


MTIP 


Maintenance Training Improvement Program 


NAMP 


Naval Aviation Maintenance Program 


NATTC 


Naval Air Technical Training Center, 
Millington, Tennessee 
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NAVAIR 


Naval Air Systems Command, Washington, 

D. C. 


NETS 


Navy Engineering Technical Services 


NPRDC 


Navy Personnel Research and Development 
Center, San Diego, California 


OJT 


On-Job -Training 


PLATO 


Program Logic for Automatic Teaching 
Operations 


SDS 


Structured Decision System 


SME 


Subject Matter Expert 


TICCIT 


Time Shared Interactive Computer- 

Controlled Information Television 
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APPENDIX C 



LIST OF RELATED WORKS 

The Related Works are categorized as being most 
applicable to one or more of the following topics: 

Computer-Based Training and Education 
Database Processing and Management 
Decision Support 

Expert System and Knowledge Technology 
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U. S. Pacific Fleet 

Naval Air Station, North Island 
San Diego, California 92135 

7. Commander, Naval Air Force 1 

U. S. Atlantic Fleet 

Norfolk, Virginia 23511 

8. Major John B. Isett 1 

Code 54IS 

Naval Postgraduate School 
Monterey, California 93943 

9. Lieutenant Colonel Gerald L. Pauler 1 

Code 55PA 

Naval Postgraduate School 
Monterey, California 93943 

10. Lieutenant Commander Galen J. Ledeboer 5 

USS Coral Sea (CV 43) 

FPO New York, New York 09550-2720 
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c.l Computer-Based Training 

applied to the U. S. Navy 
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